fix(auth): auto-switch profile on genuine exhaustion (regression from #374's false-cascade fix)#391
Open
andrei-hasna wants to merge 1 commit into
Open
Conversation
…374's false-cascade fix) #374 stopped a false weekly-limit cascade by making rolling account/rateLimits/updated snapshots display-only and gating all exhaustion bookkeeping + auto-switch to the account-verified AccountUsage read. That over- corrected: the rolling snapshot was also what drove auto-switch on GENUINE mid-turn exhaustion (it populated auth_profile_auto_switch_snapshots_by_limit_id and called maybe_auto_switch_auth_profile_for_rate_limit). After #374, when the current profile is really exhausted the switch map is empty (the ~60s authoritative heartbeat has not re-read yet and is then suppressed), and the usage-limit turn-error path only switched when a manual reset was active — so with auto-reset unavailable the failed turn just parked on a self-heal retry until the (days-away) reset instead of switching. Fix: drive auto-switch from the current profile's OWN authoritative usage-limit / 429 turn error (turn_runtime.rs on_rate_limit_error and the reset-unavailable fallback in usage_limit_reset/automatic.rs). This is authoritative for the current profile and immune to the #374 misattribution: a sibling agent's identity-less rolling snapshot never fails THIS profile's own turn. It prefers a cached account-verified exhausted window and otherwise synthesizes the trigger window from the operator's enabled auto-switch config (weekly preferred, else 5h), reusing the existing candidate selection that cycles to an Unknown-but- untried profile and bails (no loop) when every alternate is exhausted. Rolling snapshots remain strictly display-only and never seed the switch map, so #374's no-false-cascade invariant holds. Tests (tui/src/chatwidget/tests/status_and_layout.rs): - genuine_usage_limit_error_auto_switches_without_a_cached_snapshot (the regression) - rolling_snapshot_alone_never_switches_and_never_seeds_the_switch_map (#374 preserved) - genuine_usage_limit_error_does_not_loop_when_all_profiles_exhausted (graceful)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
/config"Auth profile auto-switch" toggle no longer switches profiles when the current profile is genuinely exhausted (real usage-limit / 429, e.g. "You've hit your usage limit, try again Jul 28"). The turn just parks on a self-heal retry until the (days-away) reset. This restores the legitimate switch without reviving the cross-agent false-cascade that #374 fixed.Root cause (regression from #374)
#374 (
a31719336) stopped a false weekly-limit cascade by making rollingaccount/rateLimits/updatedsnapshots display-only and gating all exhaustion bookkeeping + auto-switch behind the account-verifiedRateLimitSnapshotSource::AccountUsageread (codex-rs/tui/src/chatwidget/rate_limits.rs:314-339).That over-corrected. Before #374 the rolling snapshot was also the trigger that drove auto-switch on genuine mid-turn exhaustion: it populated
auth_profile_auto_switch_snapshots_by_limit_idand calledmaybe_auto_switch_auth_profile_for_rate_limit. After #374, on genuine exhaustion:auth_profile_auto_switch_snapshots_by_limit_id) is empty — the limit is crossed mid-turn, before the ~60s authoritative heartbeat re-reads it, and the heartbeat is then suppressed for the exhausted profile (auth_profile_popups.rs:585-591).try_auth_profile_switch_after_reset_unavailable(usage_limit_reset/automatic.rs:225-240) andmaybe_auto_switch_auth_profile_before_user_turn(rate_limits.rs:409).manual_usage_limit_reset_is_active()was true (turn_runtime.rson_rate_limit_error), so with auto-reset unavailable it just scheduled a self-heal retry for the reset instant — "it just stops."#374's own PR body flagged the follow-ups this closes ("consider verifying target capacity before switching";
choose_profile_for_auto_switchpickingUnknownblindly).Fix
Drive auto-switch from the current profile's own, authoritative usage-limit / 429 turn failure, which is authoritative for the current profile (the app server rejected this profile's own turn) and cannot be produced by a sibling agent's identity-less rolling snapshot:
try_auth_profile_switch_for_usage_limit(rate_limits.rs): prefers a cached account-verified exhausted window; when none is cached, synthesizes the trigger window from the operator's enabled auto-switch config (weekly preferred, else 5h;Nonewhen both opted out so a disabled window is never switched on) and reuses the existing candidate selection (cycles to an Unknown-but-untried profile; bails — no loop — when every alternate is exhausted). The reset instant is parsed from the error text so repeated genuine failures get distinct trigger keys and can cycle across profiles.turn_runtime.rson_rate_limit_error: the reset-unavailable branch now calls it (gated to genuineUsageLimit, not transient 429/overload), dropping the too-narrowmanual_usage_limit_reset_is_active()requirement.usage_limit_reset/automatic.rsfallback_auth_profile_switch_after_reset_unavailable: also routes through it, so the auto-reset path recovers even when the authoritative read hasn't cached the exhausted window yet.Preserves #374's no-false-cascade invariant
Rolling snapshots stay strictly display-only and never seed the switch map. The new switch is reachable only from
on_rate_limit_error/ the post-error reset fallback — never fromon_rolling_rate_limit_snapshot— so a sibling agent's foreign snapshot still cannot trigger a switch.Tests (
tui/src/chatwidget/tests/status_and_layout.rs)genuine_usage_limit_error_auto_switches_without_a_cached_snapshot— the regression: exhausted current profile + auto-switch on + another profile configured + empty map -> switches.rolling_snapshot_alone_never_switches_and_never_seeds_the_switch_map— fix(auth): stop false-positive weekly-limit cascade across auth profiles #374 preserved: rolling 100% snapshot never switches and never seeds the switch map.genuine_usage_limit_error_does_not_loop_when_all_profiles_exhausted— graceful: all alternates exhausted -> no switch, no loop (twice).Verification (Blacksmith, 16 vCPU)
codex-tuinextest suite: 3478 passed, 4 skipped, 0 failed.cargo fmt --all -- --config imports_granularity=Item --check: clean (exit 0).cargo clippy -p codex-tui --tests --all-targets -- -D warnings: clean (exit 0; only a pre-existing transitive-depproc-macro-error2future-incompat note, unrelated).Do not merge/tag yet — hand-off for 0.1.78.